Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.x](backport #5614) Adjust memory requests and limits for elastic-agent when run in Kubernetes cluster #5657

Merged
merged 2 commits into from
Oct 2, 2024

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Oct 2, 2024

What does this PR do?

This PR adjusts the default memory requests and limits for elastic-agent when it runs in a Kubernetes cluster.
The new requests and limits will be

resources:
  limits:
    memory: 1Gi
  requests:
    cpu: 100m
    memory: 500Mi

These will be adequate even in big load (tested with 95 pods per node).

Why is it important?

In latest elastic-agent versions the baseline memory consumption is higher, thus adding Kubernetes and system
integration leads to often OOM killed pods.
The previous memory limit set was 700Mb

After investigation with various number of pods we have seen that in ^8.15.* versions
the memory consumption can reach from 750 to 950 Mb.
This varies depending on the number of pods a single elastic agent has to monitor.
For example:

45 pods per node: mem consumption up to 810 MB
60  pods per node: mem consumption up to 830 Mb
75 pods per node: mem consumption up to 930 Mb
95 pods per node: mem consumption up to 990 MB

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

Disruptive User Impact

The inadequate memory limits lead to elastic-agent getting often OOM killed in newer versions.

How to test this PR locally

Deploy Elastic Agent with system and Kubernetes integration following the Kibana instructions and watch the
memory consumption of the pod with watch "kubectl top pod -n kube-system" and its status.

Related issues


This is an automatic backport of pull request #5614 done by [Mergify](https://mergify.com).

…netes cluster (#5614)

* Adjust memory requests and limits for elastic-agent when run in Kubernetes cluster

(cherry picked from commit e06e786)
@mergify mergify bot requested a review from a team as a code owner October 2, 2024 06:46
@mergify mergify bot added the backport label Oct 2, 2024
@mergify mergify bot requested a review from a team as a code owner October 2, 2024 06:46
@mergify mergify bot requested review from gizas, tetianakravchenko, blakerouse and michel-laterman and removed request for a team October 2, 2024 06:46
Copy link

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

@gizas gizas merged commit 5333ddb into 8.x Oct 2, 2024
14 checks passed
@gizas gizas deleted the mergify/bp/8.x/pr-5614 branch October 2, 2024 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants